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Abstract  —  In  the  assessment  of  system’s  developmental 
maturity,  metrics  are  used  for  making  effective  and 
efficient  decisions.  This  paper  introduces  the  combination 
of  two  metrics  used  in  systems  development,  i.e.  Systems 
Readiness  Level  (SRL)  and  Equivalent  System  Mass 
(ESM),  in  order  to  make  a  more  informed  decision  in  the 
developmental  planning  of  Exploration  Life  Support 
Systems.  We  then  expand  upon  this  approach  by  utilizing 
an  optimization  model  that  seeks  to  maximize  a  system’s 
readiness  (i.e.  SRL)  given  a  budgetary  allowance 
expressed  in  terms  of  ESM.  We  conclude  with  an 
articulation  of  this  optimization  model  utilizing  a  generic 
space  system. 

Index  Terms —  equivalent  systems  mass;  integration  readiness 
level;  system  readiness  level;  integration  readiness  level 

i.  Introduction 

Space  systems  are  neither  simple  (e.g.  technical  complexity) 
nor  inexpensive  and  it  becomes  a  continual  paradox  in  the 
trade  space  between  technical  performance  and  cost 
minimization  [1-3].  Likewise,  in  systems  engineering  there  is 
a  continual  effort  to  define  the  most  effective  and  efficient 
measures  that  will  allow  for  the  management  of  this  trade 
space.  Combined,  systems  engineering  becomes  a  critically 
core  competency  for  successful  development  of  space  systems 
[4-6],  and  is  often  seen  as  a  solution  to  the  balance  of  cost, 
schedule  and  performance  [7],  The  United  States  (US) 
National  Aeronautics  and  Space  Administration  (NASA)  has 
been  no  exception  to  the  success  of  proper  implementation  of 
systems  engineering  [4],  nor  the  failure  of  systems  because  of 
its  insufficient  implementation  [8-11].  A  key  challenge  to  the 
realization  of  systems  through  systems  engineering  is  in 
integrating  related  technical  parameters  and  components  and 
ensuring  compatibility  in  a  manner  that  optimizes  the  total 
system  definition  and  design  [12].  From  a  systems 
engineering  management  perspective,  it  becomes  critical  how 
these  challenges  are  assessed  via  a  set  of  effective  and 
efficient  metrics.  Likewise,  determining  which  technologies 
should  receive  continued  investment  in  achievement  of  a 
system’s  mission  objectives  has  strategic  and  engineering 
implications  [13]. 

The  balancing  of  technology  development  and  integration 
efforts  in  the  achievement  of  a  system’s  objectives  is  not  new 
to  NASA  or  any  organization  [14].  Yet,  the  assessment  of 
these  efforts  via  effective  and  efficient  use  of  metrics  has  been 
a  sustained  challenge  [15].  Within  NASA,  two  metrics  have 
been  researched  or  implemented  to  assess  the  developmental 


maturity  of  a  technology  or  to  determine  its  relative  impact  on 
the  system’s  mission,  i.e.  Technology  Readiness  Level  (TRL) 
and  Equivalent  System  Mass  (ESM). 

TRL  has  been  traditionally  used  within  NASA  as  an 
assessment  of  the  maturity  of  evolving  technologies  prior  to 
incorporating  them  into  a  system  or  sub-system  on  a  scale  of  1 
to  9  (9  indicating  highest  level  of  maturity).  The  original  TRL 
was  a  by-product  of  the  NASA  post- Apollo  era  as  an  ontology 
for  contracting  support  [16].  It  later  became  a  standard  metric 
for  communication  of  technologies’  developmental  status 
[17].  Other  government  agencies  and  contractors  have  since 
been  adopting  the  TRL  scale  with  specific  variations  (e.g.,  US 
Department  of  Defense,  US  Department  of  Energy,  and 
United  Kingdom  Ministry  of  Defence).  However,  TRL,  by 
definition,  can  only  refer  to  the  maturity  of  the  technologies 
but  not  the  system  as  a  whole.  For  example,  it  neglects  the 
integration  links  among  the  technologies,  which  tend  to  be 
more  complicated  and  multi-dimensional  [18].  To  address 
this  shortcoming,  Gove  [19]  and  Sauser  et  al  [20,  21] 
introduced  the  concept  of  an  Integration  Readiness  Level 
(IRL),  also  a  1  to  9  scale.  When  combined  with  TRL,  Sauser 
et  al.  [22,  23]  were  able  to  calculate  a  System  Readiness  Level 
(SRL)  and  plot  it  against  a  system  development  lifecycle  to 
evaluate  the  status  of  each  subsystem  and  the  system  as  a 
whole. 

Likewise,  traditional  design  metrics  for  space  systems,  such 
as  mass,  volume,  and  power,  have  driven  launch  costs  and 
technology  trade  options,  which  has  allowed  cost  to  become  a 
key  indicator  of  programmatic  health  for  space  systems.  To 
enable  cost  to  become  an  independent  driver,  in  the  1990’s 
ESM  was  widely  developed  and  utilized  to  evaluate  the  trade 
options  in  space  Exploration  Life  Support  (ELS)  systems  [24, 
25].  This  was  necessary  in  order  to  meet  requirements  of 
minimizing  launch  cost,  as  related  to  the  mass,  volume, 
power,  cooling  and  crew-time  needs  [26,  27].  ELS  systems 
are  those  related  to  the  long-term  habitation  and  exploration 
of  space.  In  this  context,  ESM  allowed  for  the  evaluation  of 
tradeoffs  between  two  technology  options  where  cost  was  not 
a  driver  in  the  decision  but  a  by-product.  While  ESM  has 
been  shown  to  be  a  beneficial  metric  for  making  design  trade 
decisions,  Levri  and  Drysdale  [27]  explain  that  the  tradeoff 
between  the  ESM  of  two  technology  options  may  be  so  small 
that  further  analysis  is  needed  using  a  metric  such  as  TRL. 

It  is  the  focus  of  this  paper  to  utilize  the  work  of  Sauser,  et 
al.  [23]  in  SRL  to  enhance  the  capability  of  ESM  as  Levri  and 
Drysdale  [26]  proposed  in  utilizing  TRL  as  an  additional 
decision  metric.  We  will  also  expand  upon  this  approach  by 
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utilizing  an  optimization  model  that  seeks  to  maximize  a 
system’s  readiness  (i.e.  SRL,  IRL,  and  TRL)  given  a 
budgetary  allowance  expressed  in  terms  of  ESM.  We  will 
conclude  with  an  articulation  of  this  optimization  model 
utilizing  a  generic  space  system. 

II.  Exploration  Life  Support 

The  concepts,  methods,  and  models  that  will  be  presented  in 
this  paper  can  be  utilized  with  any  space  system,  but  we  have 
chosen  to  focus  on  those  defined  as  ELS  systems.  This  is 
because  ESM  was  first  developed  and  utilized  with  ELS 
systems,  and  thus  its  relevance,  historical  comparison,  and 
availability  of  data  have  greater  inferences. 

The  Exploration  Systems  Mission  Directorate  of  the 
National  Aeronautics  and  Space  Administration  (NASA)  is 
currently  pursuing  the  development  of  the  next  generation  of 
human  spacecraft  and  exploration  systems  through  the 
Constellation  Program.  This  includes,  among  others, 
habitation  technologies  for  supporting  lunar  and  Mars 
exploration.  The  key  to  these  systems  is  the  Exploration  Life 
Support  (ELS)  system  composed  of  several  technology 
development  projects  related  to  atmosphere  revitalization, 
water  recovery,  waste  management  and  habitation.  The 
proper  functioning  of  these  technologies  is  meant  to  produce 
sufficient  and  balanced  resources  of  water,  air,  and  food  to 
maintain  a  safe  and  comfortable  environment  for  long-term 
human  habitation  and  exploration  of  space. 

The  development  of  the  ELS  system,  while  investigating 
advanced  technology  concepts,  also  builds  upon  legacy 
technologies  generated  from  prior  NASA  programs,  e.g. 
Apollo,  Space  Shuttle,  and  International  Space  Station  (ISS). 
With  varying  degrees  of  technology  maturity  in  the 
development  of  this  system,  many  challenges  arise  in  the 
development  and  testing  of  the  integration  links  among  the 
technologies  as  well  as  maintaining  an  understanding  of  the 
maturity  of  the  whole  system.  Exploration  Life  Support  is  a 
technology  development  project  under  the  NASA  Exploration 
Technology  Development  Program  (ETDP)  [28].  Aside  from 
the  development  of  system  solutions  for  atmosphere 
revitalization,  water  recovery,  waste  management,  and 
habitation  engineering,  it  has  threaded  efforts  in  systems 
integration,  modeling  and  analysis,  and  validating  and  testing 
as  well  as  being  an  integral  part  of  the  Exploration  Systems 
Mission  Directorate  of  NASA.  The  motivation  of  the 
development  of  an  ELS  system  is  to  support  the  human 
exploration  of  the  moon  and  beyond,  e.g.  Mars.  The  ELS 
project  is  guided  by  the  following  objectives  [28]: 

1)  Develop  and  mature  life  support  system  technologies  that 
meet  mission  requirements  and  fill  capability  gaps  or 
significantly  improve  the  state-of-the-art; 

2)  Develop  technologies  for  infusion  by  the  date  for  each 
vehicle’s  Preliminary  Design  Review,  approximately  six 
years  before  flight.  Provide  information  by  System 
Requirements  Review  and  at  other  interim  milestones;  and 

3)  Develop  technologies  that  are  efficient  with  respect  to 
resource  requirements  (mass,  power,  heat  rejection. 


volume,  crew  time,  consumables)  and  are  safe  and 
reliable. 

While  there  are  many  technology  options  to  achieve  the 
mission  objectives  of  an  ELS  system.  Figure  1  represents  a 
simplified  concept  architecture  of  an  ELS  system,  which  will 
be  used  to  illustrate  the  application  of  the  proposed 
quantitative  analysis  in  this  paper.  One  fundamental  variation 
in  ELS  system  options  is  the  emphasis  on  biological 
technologies  or  sub-systems  as  opposed  to 
physical/mechanical  or  chemical  processes.  In  our  example, 
we  will  assume  the  ELS  solution  has  a  biological  emphasis. 


Figure  1 .  Exploration  life  support  system  concept  architecture. 

The  technologies  depicted  in  Figure  1  are: 

Crew  Habitat:  technology  functions  include  crew 
functionality,  comfort,  and  quality  of  life  to  ensure  crew 
productivity. 

Air  Revitalization:  technology  functions  include  CO2 
partial  pressure  control;  moisture  removal;  trace  chemical 
contaminant  control;  particulate  matter  removal  and  disposal; 
atmospheric  gas  supply,  storage,  conditioning,  and 
distribution;  resource  recovery,  storage,  conditioning,  and 
recycling. 

Food  Processing:  technology  functions  include  the 
processing,  storage,  and  preparation  of  food. 

Biomass  Production:  technology  functions  include  the 
growth  of  higher  plants  for  the  purpose  of  supplying  food  and 
revitalizing  air. 

Waste  Processing:  technology  functions  include 
water/resource  recovery,  safening  and  stabilization,  disposal 
and  containment,  waste/trash  volume  reduction,  and  odor 
control. 

Water  Processing:  technology  functions  include  recovery 
of  approximately  90%  of  wastewater  to  potable  water  quality 
via  biological  or  physical-chemical  methods. 

III.  System  Readiness  Level 

Despite  the  utility  and  value  of  the  TRL  as  a  metric  for 
determining  technology  maturity  before  transitioning  into  a 
system,  TRL  was  not  intended  to  address  systems  integration 
or  to  indicate  that  the  technology  will  result  in  successful 
development  of  a  system.  Additionally,  when  TRL  is  applied 
to  components  within  a  complex  system,  the  model  of  using 
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Figure  2.  NASA  project  life-cycle  process  flow  for  ground  and  flight 
systems  with  SRL. 


plus  its  integration  to  itself.  With  the  ability  to  assess  both  the 
technologies  and  integration  elements  along  a  numerical 
maturation  scale,  the  next  challenge  was  to  develop  a  metric 
that  could  assess  the  maturity  of  the  entire  system  under 
development.  Therefore,  the  SD&ML  has  described  how 
using  a  normalized  matrix  of  pair-wise  comparisons  of  TRLs 
and  IRLs  for  any  system  under  development  could  yield  a 
measure  of  system  maturity.  SRL  is  then  calculated  as 


individual  technology  maturity  as  a  measure  of  readiness  to 
integrate  into  system  development  can  become  confounded. 
Similar  problems  also  become  apparent  with  many  other 
technology  development  metrics  and  tools  when  applied  in  a 
systems  context  [29]. 

This  lack  of  adequate  systems-level  development 
monitoring  tools  and  methodologies  has  resulted  in  several 
complex  development  programs  with  significant  shortfalls. 
Given  the  emerging  requirements  for  a  measure  of  complex 
system  readiness,  the  Systems  Development  &  Maturity 
Laboratory  (SD&ML)  was  the  first  to  propose  the  concept  of 
a  SRL  that  would  incorporate  a  TRL  and  an  IRL  for 
determining  system  lifecycle  maturity. 

Under  this  method,  the  evaluation  of  technology  using  TRL 
and  the  evaluation  of  each  integration  using  IRL  are  combined 
via  a  set  of  mathematical  formulas  (explained  in  detail  later) 
to  produce  a  holistic  assessment  where  each  technology 
within  the  system  is  weighted  according  to  all  of  its 
integrations  and  then  rolled  up  to  a  system  level.  It  is 
important  to  emphasize  that  the  SRL  is  not  a  quantitatively 
defined  rating  system,  but  is  instead  an  analytical  combination 
of  the  TRL  and  IRL  scales.  In  others  words,  the  SRL  output  is 
purely  a  function  of  the  TRL  and  IRL  inputs. 

The  SRL  scale  is  calculated  by  using  a  normalized  matrix 
of  pair-wise  comparisons  of  TRLs  and  IRLs  that  reflects  the 
actual  architecture  of  the  system.  Briefly  stated,  the  IRL 
matrix  is  obtained  as  a  symmetric  square  matrix  (of  size  n  xn) 
of  all  possible  integrations  between  any  two  technologies  in 
the  system.  For  technology  integration  to  itself,  perfect 
integration  is  assumed  (IRL=  9)  while  an  IRL  of  zero  is  used 
when  there  is  no  integration  between  two  elements.  On  the 
other  hand,  the  vector  TRL  defines  the  readiness  level  of  each 
of  the  technologies  in  the  system.  The  calculation  of  the  SRL 
has  also  gone  through  a  series  of  refinements  and  the  most 
recent  thorough  discussion  has  been  presented  by  Sauser  et  al 
[22].  This  paper  presents  another  minor  modification  by  re¬ 
naming  the  SRL  vector  (i.e.  SRL,)  as  ITRL,.  ITRL,  indicates 
the  maturity  of  technology  i  with  its  integrations  considered. 
With  a  system  comprised  of  n  technologies,  it  is 
mathematically  described  as 


ITRL\ 

'  ( IRLnTRLl+IRLl2TRL2  +  ...  +  IRLlnTRL„)hnl  ' 

[ITRL]  = 

ITRL2 

= 

(IRL,2\TRL\  +  IRL/22^^-^2  "F  •••  "F  IRLin TRLn  )  /  JTl 2 

ITRL,, 

(IRLn  lTRLl  +  IRLn  2TRL2  + . . .  +  IRLnnTRLn )  /  mn 

where  IRLl:=lRLn, 


Where  m,  is  the  number  of  integrations  with  technology  i 


„  „  r  ITRL,  +  ITRL,  + . . .  +  ITRL,, 

SRL  = - - - - - — 

n 

More  recently,  the  SD&ML  has  described  the  formulation 
and  application  of  the  SRL  as  a  metric  to  determine  the 
maturity  of  a  system  and  its  status  within  a  developmental 
lifecycle  [22,  23].  Figure  2  is  a  representation  of  the  SRL 
scale  against  the  NASA  Project  Life-Cycle  Process  Flow  for 
Ground  and  Flight  Systems  [30].  Figure  2  will  be  used  in 
later  discussions  of  the  application  of  the  SRL.  The  rationale 
behind  the  SRL  developed  by  the  SD&ML  is  that  in  the 
development  lifecycle,  one  would  be  interested  in  addressing 
the  following  considerations: 

•  Quantifying  how  a  specific  technology  is  being  integrated 
with  every  other  technology  to  develop  the  system. 

•  Providing  a  system-wide  measurement  of  readiness. 

Therefore,  SRL  is  more  than  purely  a  qualitative 
assessment.  It  requires  the  user  to  define  the  element  level 
contributions  of  the  multiple  technologies  and  integrations 
that  make  up  the  system.  In  this  way,  it  allows  managers  to 
evaluate  system  development  in  real-time  and  to  take  pro¬ 
active  measures  by  examining  the  status  of  all  elements  of  the 
system  simultaneously.  Furthermore,  the  methodology  is 
highly  adaptive  to  use  on  a  wide  array  of  system  engineering 
development  efforts  and  can  also  be  applied  as  a  predictive 
tool  for  technology  insertion  trade  studies  and  analysis  [31]. 

IV.  Equivalent  System  Mass 

ESM  was  first  defined  in  1997  as  a  metric  for  comparing 
technology  options  for  the  Advanced  Life  Support  project 
(now  referred  to  as  ELS)  [32].  It  allowed  for  the  tradeoff  of 
mass,  volume,  power,  cooling  and  crew  time  based  on  a  single 
mass  value.  The  fundamental  premise  was  that  a  mass  value 
could  be  equated  to  launch  cost  (e.g.  it  costs  $10,000  per 
pound  to  launch  a  payload  on  the  Space  Shuttle),  thus 
allowing  for  the  optimization  of  technology  options  to  achieve 
mission  objectives.  It  is  a  common  practice  in  space  systems 
development  for  mass,  as  it  relates  to  cost,  to  be  a  driver  in 
determining  the  deployment  success  of  space  products  [33- 
35].  Although  cost  as  an  independent  variable  in  the  design  of 
space  systems  has  been  prevalent  throughout  the  industry  for 
decades,  there  is  a  need  to  shift  the  emphasis  of  cost  as  a 
driver  in  the  analysis  for  engineering  space  systems  [33]. 
This  was  a  fundamental  motivation  for  using  ESM  in  lieu  of 
dollar  costs  for  technology  development  [32].  In  addition, 
ESM  provides  advantages  since  cost  estimates: 
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•  can  be  politically  sensitive; 

•  are  not  generally  released; 

•  do  not  always  include  all  cost;  and 

•  tend  to  be  complex  and  dynamic  [32]. 

ESM  allowed  for  cost  to  become  an  independent  variable 
and  did  not  have  a  direct  influence  on  a  trade  analysis.  The 
use  of  equivalent  weight  and  power  penalties  for  space 
payload  options  was  first  introduced  by  Trusch  and  Brose 
[36].  From  that  time  much  of  the  development  of  equivalent 
weight  as  an  option  metric  was  done  by  Drysdale  [32,  37] 
further  expanded  by  Levri  et  al.  [26,  27],  and  demonstrated  as 
a  decision  support  tool  by  Rodriguez  et  al.  [38]  and  Russell 
and  Carrasquillo  [24].  Accordingly,  ESM  is  calculated  as: 

ESM  =  M  +  L  +  V  *  eqV  +  P  *  eqP  +  C  *  eqC  +  CT *  D*  eqCT 

where:  ESM  =  equivalent  system  mass  value  of  the  system 
of  interest  [kg],  M  =  total  mass  of  the  system  [kg],  L  =  mass 
of  the  materials  and  spare  logistics  of  the  system  [kg],  V  = 
total  pressurized  volume  of  system  [m  ],  eqV  =  mass 
equivalency  factor  for  the  pressurized  volume  infrastructure 
[kg/m  ],  P  =  total  power  requirement  of  the  system  [kWe], 
eqP  =  mass  equivalency  factor  for  the  power  generation 
infrastructure  [kg/kWe],  C  =  total  cooling  requirement  of  the 
system  [kWth],  eqC  =  mass  equivalency  factor  for  the  cooling 
infrastructure  [kg/kWth],  CT  =  total  crew  time  requirement 
for  operation  and  maintenance  of  the  system  [CM-h/day],  D  = 
duration  of  the  mission  segment  of  interest  [day],  and  eqCT  = 
mass  equivalency  factor  for  the  crew  time  support  [kg/CM-h]. 
For  a  detailed  explanation  and  guidance  on  ESM  see  [39]. 

While  ESM  adds  value  to  the  trade  analysis  of  technology 
options  for  space  missions,  it  is  still  noted  that  it  should  not  be 
a  standalone  metric  and  additional  metrics  that  evaluate  the 
developmental  status  of  a  technology  would  be  of  added 
value.  For  example,  Czupalla,  et  al.  [25]  used  metrics  in 
reliability,  maintainability,  and  dependence  with  ESM  and 
TRL  to  perform  a  trade  study  of  spacecraft  life  support 
systems.  More  commonly,  TRL,  as  a  measure  of  technology 
maturity,  has  been  repeatedly  cited  as  a  core  metric  that 
should  be  used  with  ESM  [24,  25,  32,  38].  This  paper 
suggests  that  TRL  alone  is  not  sufficient  since  it  does  not 
measure  the  readiness  of  the  integration  elements  or  that  of 
the  system  as  a  whole.  Instead,  it  is  recommended  that  SRL 
be  used. 

In  the  next  section  we  will  expound  upon  these  advanced 
needs  in  the  effective  and  efficient  assessment  of  space 
systems  to  combine  the  two  metrics  just  described,  i.e.  SRL, 
ESM,  to  formulate  a  constrained  optimization  model  to 
demonstrate  how  these  two  metrics  can  make  a  more 
informed  decision  as  opposed  to  their  functioning 
independently. 

V.  Constrained  Optimization  Model 

SRL  was  first  used  in  a  constrained  optimization  model  by 
Sauser  and  Ramirez-Marquez  [40]  to  provide  information 
about  which  technologies  and  integration  links  to  advance  to 


which  maturity  level  such  that  the  maturity  of  the  system  is 
maximized  based  on  the  amount  of  limited  resources  made 
available  to  a  development  project.  In  this  paper,  a  similar 
optimization  model  is  applied  to  the  development  of  an  ELS 
system  to  illustrate  how  SRL  can  be  used  to  plan  its 
development.  Since  SRL  itself  is  based  on  the  TRL  and  IRL 
values  of  the  system’s  components,  it  measures  the  overall 
readiness  of  the  system  under  development.  As  such,  the 
systems  engineer  or  program  manager  who  is  concerned  with 
utilizing  the  budget  allocated  for  the  system  can  now  set 
development  goals  such  that  the  maximum  amount  of  system 
readiness  is  achieved.  In  order  to  execute  the  development 
required  to  have  maximum  SRL  value,  it  is  necessary  to  know 
how  to  utilize  the  resources  optimally.  That  is,  the  systems 
engineering  or  program  manager  must  determine  which  of  the 
system  components  should  be  matured  to  what  levels  so  that 
he  can  allocate  the  available  resources  accordingly.  To 
address  these  concerns,  we  are  proposing  Model 
ESM_SRLmax  as  an  optimization  model  whose  objective  is  to 
maximize  SRL  (a  function  of  technology  and  integration 
development)  while  keeping  the  launch  cost  (expressed  in 
terms  of  ESM)  within  an  acceptable  level.  The  general 
mathematical  form  of  the  model  follows: 

Model  ESM  SRLI|;| 

Maximize:  SRL  (TRL,  IRL ) 

Subject  to:  ESM(TRL,IRL)  <  esm 

The  matrices  IRL  and  TRL  of  the  model  contain  the 
decision  variables.  Each  of  these  variables  is  integer  valued 
and  bounded  by  (7RL,y,9)  and  (777L„9),  respectively.  That  is, 
the  TRL/IRL  for  any  component  cannot  be  below  its  current 
level  or  above  perfect  technology  or  integration  development 
(IRL  or  TRL  =  9).  The  objective  function  of  Model 
ESM_SRLmax  of  the  system  is  a  function  of  the  decision 
variables,  which  dictate  how  the  different  levels  for  both  TRL 
and  IRL  are  improved.  The  left  hand  side  of  the  inequality 
defined  by  functions  ESM  represents  the  ESM  as  a  function  of 
the  improved  technologies’  TRL  and  IRL,  and  the  right  hand 
side  indicates  the  total  amount  allowance  of  ESM  for  the 
whole  system.  Since  the  ESM  is  an  indicator  of  the  needed 
launch  cost,  the  model  tries  to  maximize  the  system  maturity 
while  under  the  ESM  allowance,  and  thus  to  meet  the  cost 
constraint. 

To  completely  characterize  the  decision  variables,  it  is 
necessary  to  introduce  the  following  transformation: 

yk  _  j1  =  k  and  %k  _  j1  IRLv  =  k  for  k=  1,. .  .9 

|f)  otherwise  [0  otherwise 

Notice  that  based  on  these  binary  variables,  each  of  the 
possible  normalized  TRL  and  IRL  in  the  system  can  be 


obtained  as:  '  9  and  ''  9 

and  ITRL,  is  transformed  to: 
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±  yteJSti)  (2*4j(2^)  2[gfaiJ[S^ 


The  model  belongs  to  the  class  of  binary,  integer-valued, 
non-linear  problems.  For  the  ELS  system  with  6  technologies 
containing  10  distinct  integrations,  and  assuming  all 
technologies  and  integrations  are  at  their  lowest  levels,  there 
can  be  as  many  as  g6+l°  potential  solutions  to  the  model. 
Evaluating  each  possible  solution  is  prohibitive  so  to  generate 
a  more  timely  optimal  solution,  a  meta-heuristic  approach 
developed  by  Ramirez-Marquez  and  Rocco  [41]  is  applied  to 
the  ELS  system.  This  approach,  called  Probabilistic  Solution 
Discovery  Algorithm  (PSDA),  has  the  capability  of  producing 
quasi-optimal  solutions  in  a  relatively  short  period  of  time. 
However,  it  must  be  mentioned  that  the  results  cannot  be 
proven  to  be  the  optimal  solution.  This  is  because  by  taking  a 
probabilistic  approach,  the  algorithm  can  only  select  subsets 
of  the  entire  feasible  set  from  which  to  find  a  solution.  Every 
time  the  algorithm  is  run,  a  different  subset  is  selected. 
Nevertheless,  prior  tests  have  indicated  that  PSDA  results 
tend  to  be  better  than  results  from  alternative  meta-heuristic 
approaches  [42]. 

As  used  in  the  solution  of  the  maximization  problem,  after 
the  algorithm  is  initialized,  it  follows  three  inter-related  steps: 


•  Strategy  Development  -  a  Monte  Carlo  simulation  is  used 
to  identify  to  what  potential  TRL  or  IRL  levels  the 
technologies  and  links  can  be  advanced  or  matured; 

•  Analysis  -  each  potential  solution  is  analyzed  by 
calculating  its  associated  SRL  and  ESM; 

•  Selection  -  through  an  evolutionary  optimization 
technique,  a  new  optimal  set  of  technologies  and 
integration  links  (with  their  corresponding  TRLs  and  IRLs 
is  chosen  (based  on  the  SRL  and  ESM  values). 


During  Strategy  Development,  based  on  the  probabilities 
defined  by  vectors  y,„  and  Yy«>  the  simulation  is  used  to 
generate  a  specified  number  (defined  by  V)  of  potential 

TRT  v  TRT  v 

designs,  «  and  »  (v=1,..,V).  For  each  technology 

k 

<  in  (the  fcth  element  of  vector  yiu)  defines  the  probability  that 
at  cycle  u,  the  TRL  of  such  a  technology  will  increase  its 

re¬ 
current  readiness  to  level  k  (i.e.  yk  =  p(yk  =i)).  Similarly,  IJU 

defines  the  probability  that  at  cycle  u,  the  IRL  between  the  ith 
and  jth  technologies  will  increase  its  actual  readiness  to  level 

yk  =  p(xk  =  A 

k  (i.e.  tv  /).  This  step  also  contains  the  stopping 

rules  of  the  algorithm.  In  essence,  the  first  rule,  which  is  used 
in  this  paper,  allows  the  user  to  set  a  specific  number  of 
cycles.  The  second  rule  dictates  the  algorithm  to  be  stopped 
once  both  vector  yitl  and  y iju  can  no  longer  be  updated  (i.e.  all 
initial  “appearance”  probabilities  are  either  zero  or  one).  In 
the  context  of  this  algorithm  a  cycle  is  understood  as  every 
time  the  value  u  is  updated. 

The  second  step.  Solution  Analysis,  implements  the 
approach  discussed  in  Sauser  et  al.  [22]  and  previously 


summarized  to  obtain  the  SRL,  and  the  ESM  of  the 
development  associated  with  each  of  the  potential  system 

design,  TRL:andIRL'M. 

The  final  step  in  the  algorithm  penalizes  the  SRL  of  the 
potential  designs  generated  in  cycle  u  whenever  they  violate 
the  ESM  constraints.  The  solutions  are  then  ranked  in 
decreasing  order  of  magnitude  with  respect  to  the  penalized 
SRL.  Then,  the  best  of  these  solutions  is  stored  in  set  K  and 
finally,  a  subset  of  size  S  of  the  ranked  feasible  solutions,  is 
used  to  update  the  probabilities  defined  by  the  vectors  y,-„  and 
y ij„.  These  new  vectors  are  re-evaluated  in  Step  1  to  check  for 
termination  or  for  solution  discovery.  Finally,  when  the 
prescribed  number  of  cycles  has  been  reached,  the  best 
solution  in  set  K  is  chosen  as  the  optimal  system  design. 

VI.  Results  and  Discussion 

For  the  generic  ESL  system  we  are  analyzing,  let  us  assume 
that  the  current  readiness  levels  of  its  components  and 
integration  links  are  shown  in  Table  I.  When  reviewing  the 
SRL  for  this  system  in  its  current  state,  the  calculations 
yielded  an  SRL  of  0.33.  Referring  to  Figure  2,  this  value 
indicates  that  this  system  should  be  in  Phase  A:  Concept  & 
Technology  Development. 

For  the  system  used  in  this  example.  Tables  2  and  3  present 
the  ESM  of  each  component  (technology  or  integration)  at 
different  maturity  levels.  For  example,  to  mature  Technology 
1  from  TRL  of  1  to  9,  its  ESM  is  estimated  to  rise  from  2,743 
to  3,234  kgs.  The  ESM  is  43,273  for  the  ELS  system  in  its 
current  status,  and  in  order  to  fully  mature  all  the  technologies 
and  integration  elements,  the  ESL  is  allowed  a  maximum 
ESM  of  44,876  without  any  amount  budgeted  for  the  usual 
management  allowance. 

To  further  explain  the  model,  we  describe  a  situation 
where,  for  example,  the  program  manager  wants  to  show  the 
customer,  in  this  case  the  Constellation  Program,  to  which 
maturity  level  or  development  stage  he  can  take  the  ELS 
system  if  he  is  given  various  ESM  allowances.  In  order  to 
answer  this,  the  PSDA  optimization  model  calculated  the 
maximum  SRL  values  when  20%,  40%,  60%,  80%  and  all  of 
the  ESM  allowance  is  allocated.  The  results  are  shown  in 
Table  IV.  For  example,  when  the  ESM  is  allowed  to  increase 
from  43,273  (current  value)  to  43,901  (utilizing  around  40% 
of  the  remaining  allowable  increase  in  ESM),  the  SRL  can  be 
increased  from  0.33  to  0.76.  This  takes  the  ELS  system  from 
Phase  A  to  a  state  where  it  would  soon  transition  from  Phase 
B:  Preliminary  Design  &  Technology  Completion  to  Phase  C: 
Final  Design  &  Fabrication. 

In  addition,  the  development  plan  which  can  achieve  the 
SRL  value  of  0.76  when  40%  of  the  incremental  ESM  is 
allocated  also  shows  that  the  subsystems  which  are  based  on 
each  technology  element  reach  their  respective  maturity  levels 
as  shown  in  Table  IV.  The  40%  case  shows  that  of  the  six 
subsystems,  three  are  ahead  (ITRLi4>6),  two  are  slightly 
behind  (ITRL2j5)  and  one,  (ITRL3)  is  close  to  the  same  level 
as  that  of  the  whole  system.  This  insight  can  become  useful 
when  the  maturity  levels  are  associated  with  systems 
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engineering  activities;  hence,  the  spectrum  of  ITRL,’s  can 
indicate  levels  of  variation  in  the  systems  engineering 
activities  which  are  needed  to  mature  the  entire  system. 

While  the  SRL  index  can  have  value  for  overall  planning, 
one  can  assess  the  developmental  maturity  of  each  technology 
and  corresponding  integrations  based  on  the  ESM  allowances 
using  Model  ESM_SRLmax.  Table  V  illustrates  the  associated 
TRL  and  IRL  levels  obtained  from  the  optimal  solution  for 
each  of  the  cases  considered.  This  is  very  important  to 
understanding  how  the  optimization  approach  can  influence 
the  developmental  maturity  of  the  individual  technologies  and 
integrations.  That  is,  the  optimal  TRL  and  IRL  levels 
obtained  from  the  model  becomes  a  guidance  tool  for  the 
systems  engineering  manager  to  better  understand  how  the 
ESM  allowances  are  impacting  maturity  of  development. 
Table  V  also  indicates  that  for  ELS,  an  80%  ESM  allowance 
still  would  not  ensure  a  fully  mature  system  because 
Technology  6  and  two  of  the  IRLs  (2,3;  2,6)  are  not 
completely  matured.  The  technology  involved  is  the  food 
processing  component  and  the  integration  elements  are  the 
ones  that  connect  the  crew  habitat  to  it  as  well  as  to  the  water 
processing  facility.  Unless  these  can  be  feasibly  matured  in 
space,  the  system  cannot  be  launched. 

It  must  be  pointed  out  that  the  design  solutions  in  Table  V 
are  calculated  using  the  budgeted  incremental  ESM. 
However,  the  solution  for  each  increasing  amount  of  allocated 
ESM  is  not  dependent  on  the  values  of  the  readiness  levels 
calculated  for  the  preceding  lower  amount  of  ESM  allocation. 
That  is,  the  algorithm  does  not  go  sequentially  from  20%  to 
40%  and  so  on,  such  that  20%  automatically  corresponds  to 
year  1  and  40%  to  year  2.  Rather,  what  the  solution  shows  is 
that  if  a  certain  %  is  allocated,  the  corresponding  technologies 
and  integrations  can  be  matured  to  such  levels  as  indicated.  It 
is  up  to  the  decision  makers  to  allocate  a  budget  for  any  given 
year  and  plan  the  development  based  on  the  available  budget. 
This  is  the  reason  why  Technology  4  can  be  matured  to  level 
9  under  20%  and  40%  ESM  allowance,  whereas  it  is  only 
matured  to  level  8  under  60%  ESM  allowance.  However,  if  a 
time -related  sequential  design  solution  is  desired,  say  for  5 
years,  a  sequential  orderly  solution  can  be  achieved  by 
following  a  recursive  manner  of  utilizing  the  ESM_SRLmax 
model.  For  example,  in  order  to  get  an  incremental  design 
solution  for  20%  and  40%  ESM  allowances  corresponding  to 
years  1  and  2  respectively,  first  execute  the  model  to  get  the 
design  solution  for  the  20%  scenario  then,  allocate  another 
20%  for  year  2  and  re-run  the  model.  That  is,  when  a  TRL  or 
IRL  has  already  been  achieved  for  a  particular  element,  it  can 
no  longer  be  de-matured  just  to  follow  the  prescribed  solution 
from  the  algorithm.  Thus,  for  the  60%  scenario.  Technology 
4  must  stay  at  TRL  9  and  not  revert  back  to  8  as  a  practical 
matter. 


system  maturity  (as  opposed  to  elemental  or  technology 
maturity),  with  accepted  space  systems  design  constraints  via 
ESM,  we  have  created  a  more  robust  and  informative 
evaluation  of  the  developmental  state  of  a  space  system.  As  a 
mean  of  indicating  the  implications  of  this  work  to  the 
practice  of  systems  engineering  of  space  systems.  Table  VI 
explains  how  TRL,  IRL,  SRL,  and  ESM_SRLmax  can  impact 
the  systems  engineering  processes  as  defined  by  the  NASA 
Systems  Engineering  Handbook  [30] 


VII.  Conclusions 

What  we  have  presented  is  a  first  step  toward  a  more 
informed  systems  engineering  management  approach  to  better 
understand  the  developmental  maturity  and  management  risks 
associated  with  deployment  of  space  systems.  By  combining 
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Table  I 


Current  readiness  levels. 


Technology 

TRL 

Technology  1 

Air  Revitalization 

5 

Technology  2 

Crew  Habitat 

4 

Technology  3 

Water  Processing 

5 

Technology  4 

Waste  Processing 

4 

Technology  5 

Biomass  Production 

5 

Technology  6 

Food  Processing 

6 

Integration 

IRL 

Integration 

IRL 

1,2 

4 

2,6 

4 

1,5 

5 

3,4 

4 

2,3 

4 

3,5 

5 

2,4 

4 

4,6 

6 

2,5 

4 

5,6 

5 

Table  II 

Cumulative  ESM  for  Technology  Elements  against  TRL 
_ (current  TRLs  in  bold) _ 


TRL 

Technology 

1 

2 

3 

4 

5 

6 

1 

2743 

2302 

3350 

1302 

2926 

17139 

2 

2835 

2551 

3489 

1385 

3074 

18499 

3 

2986 

2633 

3765 

1389 

3273 

19778 

4 

3058 

2767 

3897 

1462 

3356 

19864 

5 

3131 

2836 

3926 

1498 

3476 

20466 

6 

3212 

2873 

4004 

1510 

3526 

20988 

7 

3230 

2898 

4044 

1521 

3562 

21357 

8 

3233 

2907 

4096 

1536 

3580 

21521 

9 

3234 

2911 

4111 

1538 

3597 

21610 

Table  III 

Cumulative  ESM  for  Integration  Elements  against  IRL 

_ (CURRENT  IRLS  IN  BOLD) _ 


IRL 

Integration 

1,2 

1,5 

2,3 

2,4 

2,5 

2,6 

3,4 

3,5 

4,6 

5,6 

1 

624 

963 

1352 

371 

689 

757 

703 

241 

279 

543 

2 

679 

1017 

1477 

395 

701 

846 

765 

260 

294 

547 

3 

693 

1088 

1514 

417 

729 

896 

805 

276 

296 

585 

4 

729 

1090 

1540 

431 

744 

943 

847 

279 

300 

589 

5 

749 

1092 

1565 

438 

763 

956 

881 

290 

302 

604 

6 

761 

1116 

1581 

441 

773 

972 

901 

293 

303 

608 

7 

770 

1130 

1597 

442 

778 

973 

905 

294 

308 

612 

8 

776 

1136 

1600 

446 

784 

978 

908 

297 

310 

613 

9 

779 

1144 

1601 

448 

787 

979 

914 

299 

312 

614 

Table  IV 

Best  solution  for  ESM  increase  allowance 


Case 

ITRLi 

itrl2 

ITRL3 

ITRL4 

ITRLS 

ITRL6 

SRL 

ESM 

Current 

0.35 

0.28 

0.31 

0.33 

0.35 

0.37 

0.33 

43273 

20% 

0.50 

0.46 

0.47 

0.56 

0.50 

0.68 

0.53 

43579 

40% 

0.81 

0.69 

0.75 

0.79 

0.68 

0.83 

0.76 

43901 

60% 

0.96 

0.78 

0.89 

0.81 

0.85 

0.81 

0.85 

44221 

80% 

1.00 

0.90 

0.97 

0.92 

0.93 

0.86 

0.93 

44249 

100% 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

44876 

Table  V 

Best  design  solution  for  every  increase  in  ESM  allowance 


ESM 

Technology 

Integration 

Allowance 

1 

2 

3 

4 

5 

6 

1,2 

1,5 

2,3 

2,4 

2,5 

2,6 

3,4 

3,5 

4,6 

5,6 

Current 

5 

4 

5 

4 

5 

6 

4 

5 

4 

4 

4 

4 

4 

5 

6 

5 

20% 

5 

4 

5 

9 

7 

6 

7 

7 

4 

7 

4 

7 

4 

8 

9 

8 

40% 

5 

9 

5 

9 

9 

6 

9 

8 

5 

9 

7 

7 

8 

9 

9 

8 

60% 

8 

9 

9 

8 

9 

6 

9 

9 

7 

7 

7 

7 

8 

9 

9 

8 

80% 

9 

9 

9 

9 

9 

6 

9 

9 

8 

9 

9 

7 

9 

9 

9 

9 

100% 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

9 

8 


Table  VI 

Implications  to  NASA  SE  processes 


NASA  Systems  Engineering  Processes  T30] 

Implications  of  TRL,  IRL,  SRL,  and  ESM_SRLmax  [40] 

System  Design  Processes 

Requirements  Definition  Processes 

•  Stakeholder  Expectations  Definition 

•  Technical  Requirements  Definition 

•  TRL,  IRL,  SRL,  and  ESM_SRLmax  provide  an  enhanced  capability 
alignment  through  the  identification  of  specific  technology, 
integration,  and  system  maturities  that  can  be  used  as  a  trade  study 
tool  to  select  the  most  appropriate  technologies  and  integrations  to 
the  lowest  amount  of  risk,  cost,  and  time  and  satisfy  a  given 
customer  need. 

Technical  Solution  Definition  Processes 

•  Logical  Decomposition 

•  Design  Solution  Definition 

•  The  SRL[IRL,  TRL]  model  can  improve  customer  confidence  in  the 

SE  Manager  by  providing  a  qualification  of  system  maturity  in 
relation  to  system  and  functional  requirements.  It  can  also  provide 
improved  understanding  of  the  system’s  mission  capabilities  in 
terms  of  readiness  criteria. 

•  SRL  can  provide  an  assessment  of  maturity  at  multiple  architectural 

layers.  Any  single  SRL  assessment  contains  multiple  SRL 
assessments  from  the  SRL  vector,  which  can  provide  insight  into  the 
interdependencies  of  different  sub-functions  and  how  they  fit  within 
the  larger  architecture. 

•  Optimizing  the  system  scenarios  based  on  technology  and  integration 

maturity,  along  with  ESM,  any  system  concept  can  be  reconfigured 
with  different  technologies  and  integrations  and  the  ESM_SRLmax 
can  be  determined. 

Technical  Management  Processes 

Technical  Planning  Process 

•  Technical  Planning 

Technical  Control  Processes 

•  Requirements  Management 

•  Interface  Management 

•  Technical  Risk  Management 

•  Configuration  Management 

•  Technical  Data  Management 

Technical  Assessment  Process 

•  Technical  Assessment 

Technical  Decision  Analysis  Process 

•  Decision  Analysis 

•  Fast,  iterative  assessment  that  can  be  repeated  and  traced  during 

development.  Also,  the  ESM_SRLraax  allows  for  the  decisions  made 
during  this  activity  to  be  quantified  against  the  architecture.  This 
can  necessitate  a  valuable  exercise  in  architecture  examination  and 
creation,  which  can  allow  for  better  system  understanding  and 
(re)formation. 

•  IRL,  SRL,  and  ESM_SRLmax  allow  for  other  factors  in  addition  to 

technology  readiness  as  a  measure  of  maturity.  In  addition,  factors 
such  as  obsolescing,  by  comparative  analysis  of  multiple 
technologies  to  acquisition,  and  the  optimization  of  technology 
maturation  investment  and  transition  funding  can  be  considered. 

This  is  currently  an  area  of  future  research. 

•  SRL,  IRL,  and  TRL  provide  common  ontology  to  measure  and 

describe  acquisition  development,  system  development  and 
technology  insertion  evaluation. 

Product  Realization  Processes 

Product  Transition  Process 

•  Product  Transition 

Evaluation  Processes 

•  Product  Verification 

•  Product  Validation 

Design  Realization  Processes 

•  Product  Implementation 

•  Product  Integration 

•  IRL  reduces  the  uncertainty  involved  in  integrating  a  technology  into 

a  system  and  identifies  integration  as  a  separate,  specific  metric 
along  with  an  assessment  of  maturity  at  the  system-level. 

•  Currently  the  SRL,  TRL,  IRL,  and  ESM_SRLmax  presented  in  this 

paper  only  indicate  level  of  maturity  not  performance.  These  metrics 
are  not  intended  to  measure  system  performance  to  requirements. 
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